The room was packed at the annual Machine Learning and the Market for Intelligence conference in Toronto last week. Now in its fifth year, the lengthy name of the event matches the depth of the discussions. But one speaker and her talk stood out to me in particular: Marzyeh Ghassemi, who also happens to be a veteran of Alphabet's Verily, presented "Machine Learning From Our Mistakes." Ghassemi, an assistant professor at the University of Toronto, talked about the importance of predicting actionable insights in health care, the regulation of algorithms, and practice data versus knowledge data. But at the very end, saving the best for last, she emphasized the importance of treating health data as a resource.

Online health communities are a valuable source of information for patients and physicians. However, such user-generated resources are often plagued by inaccuracies and misinformation. In this work we propose a method for automatically establishing the credibility of user-generated medical statements and the trustworthiness of their authors by exploiting linguistic cues and distant supervision from expert sources. To this end we introduce a probabilistic graphical model that jointly learns user trustworthiness, statement credibility, and language objectivity. We apply this methodology to the task of extracting rare or unknown side-effects of medical drugs --- this being one of the problems where large scale non-expert data has the potential to complement expert medical knowledge. We show that our method can reliably extract side-effects and filter out false statements, while identifying trustworthy users that are likely to contribute valuable medical information.